Controlled temperature chain for vaccination in low- and middle-income countries: a realist evidence synthesis

Abstract Objective To evaluate the evidence describing how the controlled temperature chain approach for vaccination could lead to improved equitable immunization coverage in low- and middle-income countries. Methods We created a theory of change construct from the Controlled temperature chain: strategic roadmap for priority vaccines 2017–2020, containing four domains: (i) uptake and demand for the approach; (ii) compliance and safe use of the approach; (iii) programmatic efficiency gains from the approach; and (iv) improved equitable immunization coverage. To verify and improve the theory of change, we applied a realist review method to analyse published descriptions of controlled temperature chain or closely related experiences. Findings We evaluated 34 articles, describing 22 unique controlled temperature chain or closely related experiences across four World Health Organization regions. We identified a strong demand for this approach among service delivery providers; however, generating an equal level of demand among policy-makers requires greater evidence on economic benefits and on vaccination coverage gains, and use case definitions. Consistent evidence supported safety of the approach when integrated into special vaccination programmes. Feasible training and supervision supported providers in complying with protocols. Time-savings were the main evidence for efficiency gains, while cost-saving data were minimal. Improved equitable coverage was reported where vaccine storage beyond the cold chain enabled access to hard-to-reach populations. No evidence indicated an inferior vaccine effectiveness nor increased adverse event rates for vaccines delivered under the approach. Conclusion Synthesized evidence broadly supported the initial theory of change. Addressing evidence gaps on economic benefits and coverage gains may increase future uptake.


Introduction
In many low-and middle-income countries, standard cold chain (2-8 °C) capacity for vaccine delivery is often restricted or unreliable, 1 leading to vaccine stock-outs, increasing equipment costs and limiting the availability of vaccines in remote areas. During outreach campaigns, the use of carrier boxes and ice-packs to maintain a cold chain right to the point of vaccine administration increases time and cost, and risks vaccine damage by freezing through incorrectly placing vaccines in direct contact with ice-packs. A solution is the controlled temperature chain approach, a vaccine management protocol endorsed by the World Health Organization (WHO), which leverages the existing thermostability of certain vaccines to allow more flexibility in service delivery. 2 By providing a safe and simple protocol for storage of selected vaccines at temperatures beyond the standard cold chain, the controlled temperature chain has the potential to substantially improve vaccination equity. 3 For a vaccine to be used under a controlled temperature chain, manufacturers must demonstrate to regulators that exposure to temperatures ≥ 40 °C for a minimum three-days single planned excursion neither impedes vaccine safety nor effectiveness. Monitoring of exposures is mandated, requiring both vaccine vial monitors to measure cumulative heat exposure and peak temperature threshold indicators to measure instantaneous heat exposure. The controlled temperature chain approach is currently recommended only for special vaccine delivery strategies (e.g. births, school campaigns and outbreak response), with three vaccines currently WHO-prequalified for controlled temperature chain use. As of 2017, more than 4 million vaccines have been administered under this approach. 2,4 The WHO Controlled temperature chain: strategic roadmap for priority vaccines 2017-2020 provides a descriptive framework explaining how controlled temperature chain can lead to greater, and more equitable, immunization coverage for eligible vaccines. 2 Here we use a realist review method to synthesize evidence from controlled temperature chain experiences to identify key priorities for future research. We also establish what may be needed to promote stakeholders' interest for greater controlled temperature chain uptake in low-and middle-income countries.

Initial theory of change
We first obtained the descriptions of how the controlled temperature chain approach could lead to improved equitable immunization coverage from the Controlled temperature chain: strategic roadmap for priority vaccines 2017-2020, with descriptions supplemented from an associated commentary Objective To evaluate the evidence describing how the controlled temperature chain approach for vaccination could lead to improved equitable immunization coverage in low-and middle-income countries. Methods We created a theory of change construct from the Controlled temperature chain: strategic roadmap for priority vaccines 2017-2020, containing four domains: (i) uptake and demand for the approach; (ii) compliance and safe use of the approach; (iii) programmatic efficiency gains from the approach; and (iv) improved equitable immunization coverage. To verify and improve the theory of change, we applied a realist review method to analyse published descriptions of controlled temperature chain or closely related experiences. Findings We evaluated 34 articles, describing 22 unique controlled temperature chain or closely related experiences across four World Health Organization regions. We identified a strong demand for this approach among service delivery providers; however, generating an equal level of demand among policy-makers requires greater evidence on economic benefits and on vaccination coverage gains, and use case definitions. Consistent evidence supported safety of the approach when integrated into special vaccination programmes. Feasible training and supervision supported providers in complying with protocols. Time-savings were the main evidence for efficiency gains, while cost-saving data were minimal. Improved equitable coverage was reported where vaccine storage beyond the cold chain enabled access to hard-to-reach populations. No evidence indicated an inferior vaccine effectiveness nor increased adverse event rates for vaccines delivered under the approach. Conclusion Synthesized evidence broadly supported the initial theory of change. Addressing evidence gaps on economic benefits and coverage gains may increase future uptake.
piece. 2,5 Once obtained, we articulated these descriptions as a theory of change following a previously published context-mechanism-outcome construct. 6,7 To ensure representativeness and accuracy of the articulated theory of change to source descriptions, we consulted the WHO Controlled Temperature Chain Working Group. Outcomes included in the theory of change were limited to implementation and excluded manufacturing considerations.
We commenced the evidence synthesis once a consensus on the initial theory of change had been reached.

Evidence search
We searched MEDLINE®, EMBASE®, CINAHL and Web of Science (all databases) using the key terms presented in Box 1, up to 7 April 2022. Targeted online searches, reference combing and contacting the WHO working group for additional evidence complemented the search. We applied no language restrictions.
Studies using either controlled temperature chain or controlled temperature chain-relevant approaches (e.g. planned storage of non-controlled temperature chain-approved vaccines beyond the standard cold chain) were eligible for inclusion. We excluded perspectives from key stakeholders (e.g. commentary pieces, laboratory-based studies) or studies that were not based on an implementation experience. Economic modelling studies were considered eligible if costs were ascertained from a controlled temperature chain or controlled temperature chain-relevant implementation experience.
Two authors subjectively evaluated whether studies provided evidence on at least one aspect in the theory of change construct and the rigour of the evidence. To assess rigour in studies with sufficient methodological description, we used three quality assessment checklists; (i) Cochrane Effective Practice and Organization of Care for quantitative studies; 8 (ii) Critical Appraisal Skills Programme for qualitative studies; 9 and (iii) Consensus on Health Economic Criteria list for economic evaluations. 10 Across all checklists, we classified quality of evidence on a three-tier (yes/no/unclear or high quality/low quality/unclear) scale.

Extraction and synthesis
Data extraction was thematic, guided by context-mechanism-outcome constructs within the initial theory of change, and supplemented by relevant categories from the WHO Supporting the Use of Research Evidence checklist. 11 We did not apply any saturation threshold. Two authors initially extracted the text verbatim from source documents, and subsequently aggregated the text to identify key overlapping or contrasting concepts.
We employed narrative methods for the synthesis, 12,13 allowing for aggregation of quantitative and qualitative findings. Two authors completed the synthesis, then it was cross-checked by other authors for consensus. Levels of supporting evidence for mechanisms and outcomes were subjectively determined, informed by quality and quantity of included studies. Evidence was deemed strong if supported by high-quality evidence and results repeated across multiple experiences. The synthesis was iterative and we revised the initial theory of change if supported by identified evidence.
The synthesis was adherent with reporting standards for realist syntheses 14 and, where applicable, preferred reporting items for systematic reviews and meta-analyses. 15 We registered the synthesis on OSF (https:// osf .io/ a3z6s).

Results
The initial theory of change shows that due to the historic reliance on the standard cold chain, awareness of controlled temperature chain and relevant use cases are needed to drive demand and uptake (domain 1). Once safely and effectively implemented within special vaccination activities (domain 2), controlled temperature chain would enhance efficiency (domain 3) and equity of coverage (domain 4; Fig. 1).
Six articles had quality assessment domains at a high risk of bias or with an unmet criterion. 20,22,28,33,38,41 Most included studies had one or more domains where evidence quality was unclear. Further details are available in the data repository. 49

Demand and uptake
The key context identified for the demand and uptake of the approach was appreciation of disease burden and a need to overcome cold chain limitations to vaccinate hard-to-reach populations. Appreciation of these standard cold chain limitations were reported in many implementation experiences, 4,[20][21][22][23]26,32,37,42 but by only 24% (6/25) of national stakeholders in a survey assessing interest for controlled temperature chain for prequalified HepB vaccines. 35 We anticipated vaccine damage due to cold chain failures (freezing or heat exposure) to be a contextual driver of controlled temperature chain demand. While in two surveys, national and global stakeholders saw utility of such approach to avert these problems, 41,46 vaccine damage as a driver of demand was only cited in four unique implementations. 16,17,21,34,36 Some evidence supported the awareness of controlled temperature chain as a Box 1. Key terms used to identify studies on controlled temperature chain for vaccination • "controlled-temperature chain" or "controlled temperature chain" or "out-of-the-cold-chain" or "out-of-cold-chain" or "outside the cold chain" or "outside cold chain" or thermostable • AND • vaccine or vaccination or immuni$e or immuni$ation or HPV or OCV or Hepatitis B or Tetanus or Birth Dose or MenAfriVac or Meningitis Systematic reviews Controlled temperature chain for vaccination Christopher P Seaman et al.
mechanism to drive demand and uptake among policy-makers. Three surveys indicated that 72% (18/25) to 75% (21/28) of national and global policy-makers showed a demand for controlled temperature chain use. 19,41,46 Awareness raised through policy adaptations, including WHO endorsement for controlled temperature chain-relevant storage of the HepB birth dose, predicated uptake in two implementations. 20,22 Awareness via endorsement from implementation partners was noted to influence uptake by health ministries in oral cholera vaccine and tetanus toxoid-containing vaccine experiences. 37,40 There was limited evidence on identification of credible and beneficial use cases for controlled temperature chain vaccines, a mechanism required to drive sustained demand by policy-makers. We identified a clearly defined use case for meningitis A conjugate vaccine across four unique experiences: campaigns in remote sub-Saharan Africa with limited or no access to a cold chain. 4,[30][31][32][33] For other vaccines the evidence was emerging or yet to be determined. For example, controlled temperature chain facilitated self-administration of a second oral cholera dose or vaccination to be completed alongside more traditional cholera control strategies. 35,38,40,42 We could not identify agreed controlled temperature chain use cases for the HepB vaccine despite 10 controlled temperature chainrelevant implementations. [16][17][18][20][21][22][23][24][26][27][28][29] Even with a controlled temperature chain-licensed vaccine and awareness of relevant coverage barriers in HPV vaccination efforts, 43,48 we only identified a single implementation. 34 For vaccines with no defined use case, 20% (36/183) of respondents across two surveys saw the controlled temperature chain as a fall back mechanism for transient cold chain breaks. 19,41 Sustained demand, via controlled temperature chain-licensure, varied across controlled temperature chain-relevant experiences; noted in three oral cholera vaccine experiences but only in a single HepB vaccine experience. 22,38,40,42 In at least four experiences, controlled temperature chain-relevant vaccine storage enabled integration of vaccination into existing health programmes. 24,28,29,40,42 However, fears of higher vaccine procurement costs, training feasibility and use of controlled temperature chain leading to poor cold chain practices dampened stakeholders' demand for the controlled temperature chain. 19,41,46 Theory of change domain 2

Safe and compliant use
Based on the evidence synthesis, we reframed this domain to focus on evidence of safe and compliant controlled temperature chain implementation (Fig. 3).
Within the context of special vaccination programmes, presence of temperature exposure monitoring technologies mediated safe and compliant vaccine use, including when used by health volunteers or by community members self-administering vaccines. 28,29,35,38,39 Ten studies relied on vaccine vial monitors measur- Note: We derived this construct from descriptions contained in the Controlled temperature chain: strategic roadmap for priority vaccines 2017-2020. 2 Downward arrows indicate link of context-mechanism-outcome within each theory of change domain. Horizontal arrows link domains to each other within the overall theory of change.

Systematic reviews
Controlled temperature chain for vaccination Christopher P Seaman et al.
ing cumulative heat exposures as the sole indicator of vaccine integrity. [18][19][20][21][22]36,[38][39][40]42 Five experiences used peak temperature threshold indicators to measure instantaneous heat exposures, 4,16,17,31,34 while digital thermometers or comparison to ambient temperatures were also used in four experiences. 22,23,36,47 In addition, use of monitoring forms to track vaccine use and exposures over an implementation was reported in four experiences. 4,33,34,36 Strong and consistent evidence demonstrated that training of vaccinators in safe and compliant use of controlled temperature chain was feasible and nononerous. Training was often less than a day in duration or integrated among other programmatic activities. 16,22,29,33,34,47 Training encompassed vaccine vial monitor interpretation, controlled temperature chain protocol awareness and use of peak temperature threshold indicators. While methodological rigour in evalu-ation of training success varied across experiences, and some protocols deviated from currently accepted controlled temperature chain standards, 16,17,36,42 good compliance was consistently reported across all professional levels. Supervision, facilitated by implementation partners 4,34,42 or project managers, 28,29 was feasible and important for maintenance of correct practices and maximizing benefits of the controlled temperature approach. Examples included provision of real-time feedback to maximize efficiency during a meningitis A conjugate vaccine implementation in Benin, 4 and identification and rectification of unsafe vaccine disposal by health volunteers in Papua New Guinea. 29 Furthermore, supervision and training were seen as potential mechanisms to catalyse use of the controlled temperature approach where permitted in Kiribati. 24 Further, no evidence suggested that the approach adversely affected other cold chain practices; however, we could only discern this interpretation from two implementations. 4

Programme efficiency
The driver of improving programme efficiency and timeliness was existing inefficiencies in maintaining a standard cold chain for special vaccination programmes. Challenges and costs associated with providing cold chains for timely service delivery were described in studies on meningitis A conjugate vaccine and oral cholera vaccine experiences. 4,30,31,40,42 Controlled temperature approaches were considered a more efficient alternative, linking to a demand for uptake. This feedback loop -efficiency of controlled temperature chain to overcome cold chain challenges leading to increased demand -is now reflected within the revised theory of change (Fig. 3).
This feedback loop was also supported by some evidence of a vaccinator preference (implying a demand) for controlled temperature chain or controlled temperature chain-relevant approaches over the standard cold chain. This preference was driven by efficiency related mechanisms, namely reported timesavings from no longer having to prepare, carry and replenish ice-packs during outreach vaccination efforts. 4,34,36 A controlled temperature chain was generally evaluated as not adding any additional complexity to service provision, except in one experience where the increased frequency of vaccine replenishment was reported to increase workload. 20 Little evidence existed in quantifying reduced costs of vaccination under the approach. Two implementations using HepB birth doses showed controlled temperature chain-relevant use of compact pre-filled auto-disable devices was more efficient than standard cold chain approaches, but savings were driven by devices-mediated task-shifting and waste reduction and could not be directly attributed to the controlled temperature chain. 18,28 As prequalified controlled temperature chain vaccine experiences were limited, we could not assess trade-offs between higher vaccine costs and reduced cold chain costs, a concern raised in interviews with key stakeholders. 19 that a proportion of cold chain costs were avoided when using the controlled temperature chain. 30,45 However, a direct comparison in one experience showed no incremental cost differences between the two approaches. 30 Evidence supported two new mechanisms by which controlled temperature chain enhanced vaccination efficiency and timeliness. First, the approach enabled more rapid delivery of vaccination to target populations. 18,28,29,38,40,42 Second, the approach was not associated with any additional vaccine wastage when compared with cold chain, 30,36 and where any measurable wastage of vaccines stored beyond the cold chain occurred, it resulted from stock management and microplanning failures. 20,34 We revised the key outcome for this domain in two elements after evidence synthesis (Fig. 3). Time-savings were reported, but not quantified, while no implementation study quantified cost savings. Credible but theoretical cost savings attributable to controlled temperature chain were extrapolated from modelling; 30,45 and where cost savings were reported, causality could not be disentangled from the use of compact pre-filled auto-disable devices. 18,28 Theory of change domain 4

Equitable vaccination coverage
The ability of the controlled temperature chain approach to improve equitable vaccination coverage was supported by promising evidence; however, experiences designed to quantify coverage gains were restricted to HepB birth vaccination. [20][21][22]26,27 Coverage benefits varied by implementation setting: in Lao People's Democratic Republic, coverage gains were greatest for births in health facilities; 22 whereas in rural China, the approach was most beneficial for timely coverage of home births. 21 In some studies, observed coverage gains due to the approach were cited as motivation for uptake in other experiences, [20][21][22] forming a feedback loop between equitable coverage gains and controlled temperature chain uptake. This feedback loop is now reflected in the revised theory of change (Fig. 3). Across controlled temperature chain and controlled temperature chain-relevant HPV, oral cholera and meningitis A conjugate vaccine experiences, reported high levels of coverage were unlikely to be achieved unless vaccines were stored beyond the cold chain. 4,32,35,40 We found no evidence of the approach increasing adverse event rates or reducing vaccine effectiveness. 17,[19][20][21]23,31,[36][37][38]44 Further, researchers for two studies in Viet Nam hypothesized that controlled temperature chainrelevant storage of the HepB birth dose have enhanced immunogenicity due to prevention of freezing. 23,26

Discussion
We identified a credible evidence base that broadly supported the initial theory of change; however, synthesis identified some key refinements. Evidence supported feasibility of safe controlled temperature chain integration into vaccination programmes, with robust evidence showing complicit and safe use by vaccinators. However, clearly defined use cases for most controlled temperature chain-eligible vaccines were lacking. Future research priorities to promote uptake of controlled temperature chain approach should include economic evaluations and studies to quantify equitable coverage gains.
Meningitis A conjugate vaccine delivery under the controlled temperature chain was only implemented in the African Region, probably due to the fact that the vaccine is designed for use in the sub-Saharan meningitis belt. 50 Implementations of controlled temperature chain-relevant approach for the HepB birth dose were limited to the Western Pacific and South-East Asia regions. One reason may be the prominence of vertical transmission of HepB in these regions as compared with the African Region, 51 and a generally low adoption and scale-up of HepB birth dose in the African Region. 52 Another reason may be national and regional frameworks endorsing controlled temperature chain-relevant birth dose use in Western Pacific and South-East Asia regions, 24

Systematic reviews
Controlled temperature chain for vaccination Christopher P Seaman et al.
fashion. Observed compliance with protocols was high across a wide range of health-education levels, including community members and volunteers. In addition, despite stakeholder concern, no evidence suggested practice of the approach led to miscreant cold chain practices. Evidence on safe and compliant use should encourage policy-makers to explore use cases where controlled temperature chain is the only option, such as storage of the HepB birth dose in remote locations or self-administered oral cholera vaccine. These use cases could both generate a sustained demand and facilitate more equitable, or timelier, vaccination coverage.
One key refinement made to the initial theory of change, shown as a feedback loop, was that efficiency of controlled temperature was driving demand. Decision-makers thought the approach overcame the problems of maintaining the cold chain in challenging circumstances and vaccinators preferred the decreased workload compared with standard cold chain. While few studies described averted freeze damage due to controlled temperature chain during implementations, we note that freeze damage is a common occurrence in many cold chains and poorly recognized by service providers, [54][55][56] and performance gains on this aspect may have gone unreported in experiences. However, policy-makers hesitated regarding the potentially higher vaccine prices for a prequalified controlled temperature chain vaccine. We found no real-world implementation evidence to counter this hesitancy; rather, any evidence of cost-effectiveness (derived through gains in vaccine delivery efficiency) are currently derived from extrapolations or theoretical modelling. 30,45,57,58 Given noted hesitations, future research should cover this area to help generate demand for the controlled temperature chain.
Another key refinement made to the theory of change was identification of a feedback loop between demand and eq- Overall cost-effectiveness is improved even with increased vaccine commodity cost

Fig. 3. Revised context-mechanism-outcome construct for the theory of change of controlled temperature chain for vaccination in low-and middle-income countries
The controlled temperature chain is cost saving Reduced reliance on cold chain, transport and lower staff burden Evidence of safety and effectiveness of ambiently stored vaccines Note: Text in italics is revision of the initial theory of change in Fig. 1. Downward arrows indicate link of context-mechanism-outcome within each theory of change domain. Horizontal arrows link domains to each other within the overall theory of change.

Systematic reviews
Controlled temperature chain for vaccination Christopher P Seaman et al.
uitable coverage gains. We observed that coverage gains attributable to controlled temperature chain-relevant storage were a driver of uptake in some experiences, and may provide a sense of confidence that the approach will be beneficial. However, more studies which quantify the direct coverage gains attributable to a controlled temperature chain approach are required. Demand for controlled temperature chain may increase if a causal link can be established between coverage gains and the approach, and not from an enhanced effort as occurs in a pilot study context.
Our synthesis has some key limitations. First, we limited our scope to uptake and implementation experiences and did not seek to include experiences of manufacturers or developers. While important and a potential avenue for future research, manufacturers' willing-ness to develop or relicense vaccines for the controlled temperature chain will likely depend upon demand, a key focus of this synthesis. Second, we cannot exclude publication bias from synthesis findings. We did not identify any failed experiences. If these failures occurred, the possibility exists they did not get reported. Finally, realist methods are inherently subjective, and findings could be influenced by researcher perspectives.
In an attempt to counteract this, results were frequently communicated to research commissioners and other experts for cross-checking.
Synthesis of evidence from controlled temperature chain approaches broadly supported the existing theory of change. Credible evidence demonstrated the overall feasibility of controlled temperature chain to improve equitable vaccination coverage in low-and middle-income countries, as well as supporting that integration of the approach into vaccination programmes is safe. Future research should conduct use case studies for eligible vaccines and quantify the economic and attributable coverage benefits of the approach in a range of health systems. ■ Funding: CPS receives support through an Australian government Research Training Program scholarship.